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Assessing  prediction  confidence  and  enabling  its  use  as  a  decision-making  metric  for 
autonomous  model  fidelity  selection  is  essential  to  the  USAF’s  vision  of  a  ‘Digital  Twin’  as  a 
viable  approach  for  condition-based  fleet  management  by  tail  number.  Significant  strides 
have  been  made  in  modeling  complex  interactions  of  the  multi-physics,  fluid-thermal- 
structural  coupling  applicable  to  hypersonic  flow  conditions.  However,  validation  of  these 
models  remains  a  challenge  due  to  limited  experimental  data  for  hypersonic  conditions.  This 
research  addresses  quantifying  errors  and  assessing  the  confidence  in  aerodynamic  pressure 
and  heating  predictions  for  a  spherical  dome  protruding  from  a  flat  ramp.  Well- 
characterized  aerothermal  test  data  from  hypersonic  wind  tunnel  experiments  are  used  to 
calibrate  uncertain  model  parameters  and  quantify  errors  through  Bayesian  techniques.  A 
Bayesian  hypothesis  testing-based  confidence  metric  is  employed  to  compare  the  accuracy  in 
various  model  predictions.  A  model  selection  study  is  performed  for  1st-,  2nd-,  and  3rd-order 
piston  theories.  The  results  showed  that  the  greatest  confidence  in  model  predictions  does 
not  necessarily  correspond  to  the  highest-order  model. 
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Nomenclature 

Bayes  factor 

Bayesian  hypothesis  testing-based  confidence  metric 
diameter  of  the  spherical  dome 
model  error 

height  of  the  spherical  dome 
Mach  number 
aerodynamic  pressure 
dynamic  pressure  ( pU 2 /2) 
aerodynamic  heat  flux 

equivalence  ratio  (fuel-to-air  ratio  divided  by  stoichiometric  fuel-to-air  ratio) 
main  effect  sensitivity  index 
total  effect  sensitivity  index 
temperature 

transverse  panel  displacement 
model  prediction,  location  along  dome 
observed  data 

oblique  shock  angle  relative  to  freestream 
uncertain  input  parameters 
ratio  of  specific  heats 
mean 

probability  density  function 
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6  =  panel  inclination  angle  to  freestream 

p  =  density 

<j  =  standard  deviation 

Subscripts 
aw 
e 
i 

true 
pred 
w 

1 

3 

4 

Superscripts 

fp  =  flat  plate 

s d  =  spherical  dome 

=  flow  properties  evaluated  at  Eckert’s  reference  temperature 

I.  Introduction 

DVANCES  in  computational  capability  and  model  fidelity  have  inspired  the  USAF  to  develop  a  plan  for 
creating  a  Digital  Twin  for  every  aircraft  platform.  The  Digital  Twin  vision  is  to  enable  condition-based  fleet 
management  by  tail  number  through  numerical  simulation  of  the  structural  response  to  the  same  flight  spectrum  as 
experienced  by  the  physical  system.  That  is,  the  Digital  Twin  must  be  capable  of  integrating  extreme  environmental, 
coupled  loading  with  advanced  damage  initiation  and  accumulation  models  for  life  prediction.  This  is  especially  the 
case  for  ultra-high  performance  platforms,  such  as  reusable,  air-breathing  hypersonic  vehicles,  since  full-scale 
testing  of  the  various  disciplines  is  often  impractical.1  Unfortunately,  obtaining  the  needed  long  time  histories  from 
these  complex  models  usually  creates  an  intractable  computational  problem.2  Therefore,  it  is  critical  for  a  hypersonic 
twin  to  be  capable  of  autonomously  selecting  between  competing,  variable-fidelity  models  for  efficient  and  accurate 
representation  of  coupled  fluid- thermal-structural  interactions.  However,  errors  inherently  exist  in  all  computational 
model  predictions  due  to  imperfect  knowledge  and  physical  variability  in  the  system,  model  order  reduction, 
assumptions  and  approximations,  and  the  limited  experimental  data  available  for  model  validation.  This  initial  phase 
of  a  broader  research  objective  is  focused  on  quantifying  the  errors  in  existing  aerothermal  model  predictions3,4 
corresponding  to  a  set  of  experiments  in  a  high-temperature  wind  tunnel.5  The  methods  implemented  in  this  paper 
will  be  used  as  a  basis  for  expanding  error  quantification  and  prediction  confidence  assessments  to  a  coupled 
aerothermoelastic  model. 

Aircraft  structures  exposed  to  extreme  environments  are  subjected  to  coupled  aerodynamic,  thermal,  and 
acoustic  loading.2,3,6"12  Neglecting  these  interactions  can  lead  to  gross  errors  in  model  predictions.10'15 
Aerothermoelastic  aircraft  structures  can  be  modeled  at  multiple  levels  of  fidelity  for  structural  and  thermal  effects. 
However,  there  are  limitations  in  computational  resources,  which  make  the  degree  of  model  fidelity  and  the  level  of 
coupling  necessary  for  a  particular  problem  play  a  key  role  in  the  computational  tractability  of  the  model  used.  For 
example,  aerodynamic  pressure  and  heating  could  be  calculated  using  a  computational  fluid  dynamics  (CFD)  model, 
or  the  decision  could  be  made  to  use  reduced-order  models  or  the  simpler  piston  theory  and  Eckert’s  reference 
enthalpy  method,  respectively.3,4,12,16  Substantial  research  has  been  performed  on  investigating  the  model 
components  for  the  physics  of  a  coupled  aerothermoelastic  panel  and  the  solution  procedures  for  both  quasi-static 
and  dynamic  solutions.2,3,6"12  However,  the  current  state  of  the  art  focuses  on  deterministic  calculations  with  limited 
uncertainty  analysis.  Lamorte  et  al.  investigated  the  implementation  of  a  stochastic  collocation  approach  for 
propagating  uncertainty  in  aerothermoelastic  analysis.17  Related  work  expanded  on  uncertainty  propagation  in 
aerothermoelastic  analysis  for  hypersonic  vehicles  with  emphasis  on  assessing  the  impact  of  aerothermoelastic 
deformation  on  aerodynamic  heating.13  Culler  et  al.  also  identified  two-way  coupling  between  structural  deformation 
and  aerodynamic  heating  as  an  important  consideration  in  modeling  an  aerothermoelastic  panel.11  These  efforts 
underscore  the  importance  of  understanding  the  uncertainty  in  a  coupled  aerothermoelastic  model;  however,  many 
questions  remain  about  the  significant  deterministic  and  stochastic  sources  of  uncertainty  and  how  to  assess  the 
confidence  in  model  predictions. 


=  adiabatic  wall 
=  edge  of  boundary  layer 
=  variable  index,  position  along  spherical  dome 
-  true  value  of  x 
=  model  prediction  of  x 
=  wall,  aerodynamic  surface 
=  freestream  flow 
=  flow  at  the  leading  edge  of  panel 
=  flow  at  location  of  interest  along  the  panel 
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Obviously,  uncertainty  is  prevalent  in  any  coupled  system  due  to  physical  variability,  sparse  data,  and  modeling 
errors.  Physical  variability  is  inherent  in  fluid- thermal- structural  interactions  through  variations  in  material 
properties,  geometry,  boundary  conditions,  and  load  interactions.  Uncertainty  also  exists  in  the  experimental  data 
used  for  model  calibration  and  validation  due  to  limited  availability  and  difficulties  in  creating  an  experimental 
environment  capable  of  fully  validating  the  model.  This  is  particularly  the  case  for  hypersonic  aircraft  structures 
exposed  to  extreme  environments.  Finally,  the  aerothermoelastic  model  prediction  has  both  model  form  error  and 
numerical  errors.  In  this  context,  model  form  error  encompasses  the  errors  in  representing  the  physical  system  with  a 
particular  model.  Numerical  errors  include  errors  from  sampling,  discretization,  coupled  solution  procedures,  and 
other  mathematical  approximations.  In  the  presence  of  these  various  uncertainty  sources,  engineers  are  challenged 
with  resource  allocation,  uncertainty  quantification,  model  calibration,  and  model  validation.  An  attractive  option  for 
integrating  errors  and  reducing  uncertainty  when  limited  data  is  available  is  the  use  of  Bayesian  techniques.18,19 
These  techniques  provide  the  statistical  information  to  validate  models  and  quantify  the  confidence  in  their 
predictions.  Integrating  statistical  distributions  and  observed  data  in  a  systematic  Bayesian  framework  for  capturing 
interactions  of  uncertainty,  model  predictions,  and  experimental  data  is  achieved  through  a  Bayes  network.18,19  A 
Bayes  network  is  a  versatile  tool  for  performing  model  validation,  sensitivity  analysis,  assessing  model  extrapolation 
capability,  and  determining  experimental  and  computational  resource  allocation.5,18"20  Bayes  networks  enable  the 
fusion  of  various  forms  of  information,  such  as  model  predictions,  experimental  data,  subjective  information,  errors, 
and  data  uncertainty.21 

The  current  research  is  part  of  a  longer-term  initiative  to  create  a  framework  for  integrating  various  sources  of 
uncertainty  in  a  coupled  hypersonic  structural  simulation  and  assessing  the  confidence  in  model  predictions.  This 
study  lays  the  groundwork  for  the  previously  discussed  Bayes  network  for  a  coupled  aerothermoelastic  system.  The 
four  primary  objectives  of  this  paper  are:  1)  perform  sensitivity  analysis  to  identify  significant  variables,  2)  calibrate 
uncertain  model  inputs  and  quantify  model  errors  using  experimental  data  with  Bayesian  updating,  3)  validate  the 
predictions  using  a  Bayesian  hypothesis  testing-based  confidence  metric,  and  4)  use  the  confidence  metric  to  make 
decisions  for  model  selection.  The  existing  models  considered  in  this  study  correspond  to  aerothermal  tests 
performed  by  NASA  on  spherical  domes  protruding  from  a  flat  ramp  into  Mach  6.5  flow.5  These  comprehensive  and 
unique  experiments  were  conducted  in  1986  by  Glass  and  Hunt  in  the  Langley  8-foot  High-Temperature  Tunnel 
(HTT)5,  and  the  tests  are  still  used  for  validation  purposes  in  numerous,  on-going  research  efforts.3,4 

The  outline  of  the  paper  is  as  follows.  Section  II  describes  the  coupled  aerothermoelastic  problem,  as  well  as  the 
simplified  aerothermal  model  corresponding  to  the  Glass  and  Hunt  HTT  experiments.  In  Section  III,  the  model  error 
for  the  aerothermal  problem  is  analyzed  in  four  steps.  First,  the  input  uncertainties  are  defined,  along  with  sensitivity 
analysis  for  the  aerodynamic  pressure  and  heating  calculations.  Next,  the  uncertain  model  parameters  and  errors  are 
calibrated  using  Bayesian  updating  with  data  from  Glass  and  Hunt5  experiments.  Subsequently,  the  calibrated 
uncertainty  and  errors  are  used  for  validation  with  experimental  data  from  a  different  spherical  dome.  Finally,  a 
Bayesian  hypothesis  testing-based  confidence  metric  is  used  to  compare  several  different  model  predictions  with  the 
experimental  data. 


II.  Aerothermal  Model  Definition  and  Experiments 

Consider  a  panel  section  on  the  forebody  of  a  representative  hypersonic  vehicle  configuration,  as  shown  in  Fig. 
I.3  As  the  vehicle  is  subjected  to  hypersonic  flow  (location  ‘1’),  an  attached  oblique  shock  is  created  at  the  forebody 
leading  edge.  This  results  in  aerodynamic  pressure  at  the  area  of  interest  (location  ‘4’),  causing  elastic  deformation 
of  the  panel,  which  feeds  back  to  alter  the  aerodynamic  pressure  on  the  panel.  This  is  commonly  referred  to  as  the 
aeroelastic  portion  of  the  coupling.  The  panel  is  also  subjected  to  aerothermal  effects  from  aerodynamic  heating. 
Naturally,  this  aerothermal  component  is  coupled  to  the  aeroelastic  component,  since  a  change  in  the  temperature  of 
the  structure  causes  additional  deformation,  which  in  turn  further  alters  both  the  aerodynamic  pressure  and  the 
aerodynamic  heating.  Figure  2  schematically  illustrates  these  interactions  as  the  coupling  of:  aerodynamic  pressure, 
aerodynamic  heating,  heat  transfer,  and  structural  deformation. 
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Leading  edge  shock 
2 


1  =  freestream 

2  =  behind  oblique  shock 

3  =  leading  edge  of  panel 

4  =  point  of  interest  along  panel 


Figure  1.  Representative  hypersonic  vehicle  structure  with  aerothermoelastic  panel3 


Figure  2.  Aerothermoelastic  model 


Modeling  these  interactions  can  be  critical  for  accurately  predicting  the  structural  response  under  hypersonic 
flow  conditions.  However,  due  to  the  complexity  of  the  problem,  presumably  less  important  couplings  are  typically 
neglected  in  favor  of  a  more  tractable  solution.  On-going  research  is  investigating  models  for  each  of  the  four 
aerothermoelastic  system  components,  as  well  as  their  integration  to  study  the  importance  of  different  types  of 
coupling.3,4,13 

In  order  to  validate  these  models  and  quantify  the  confidence  in  their  predictions,  experimental  data  from  this 
extreme,  hypersonic  environment  is  required.  A  candidate  for  validation  data  under  these  conditions  are  the 
experiments  performed  by  Glass  and  Hunt,  in  which  a  series  of  tests  were  conducted  in  a  hypersonic  wind  tunnel  to 
investigate  the  aerodynamic  loads  on  deformed  surface  panels.5  To  simulate  a  deformed  panel,  a  rigid  spherical 
dome  protuberance  was  mounted  on  a  flat  panel  holder.  While  the  use  of  rigid  domes  removes  the  aeroelastic 
coupling,  valuable  aerothermal  data  was  obtained.  The  8-foot  High-Temperature  Tunnel  can  simulate  up  to  Mach  7 
flow  at  an  altitude  between  25  and  40  km  for  up  to  2  minutes  by  combusting  a  mixture  of  methane  and  air.  The  flow 
conditions  for  the  tests  of  interest  had  a  turbulent  boundary-layer  at  the  panel  location,  and  the  panel  holder  had  a 
sharp  leading  edge,  similar  to  the  representative  hypersonic  vehicle  depicted  in  Fig.  1. 

The  experiments  performed  by  Glass  and  Hunt  used  a  flat  plate  specimen  to  record  the  aerodynamic  pressure  and 
heat  flux  at  the  center  of  the  plate  as  a  reference.  In  addition,  spherical  pressure  and  thermal  domes  with  a  diameter 
of  35.6  cm  and  the  three  HID  ratios  shown  in  Table  1  were  instrumented.  Table  1  also  summarizes  the  freestream 
conditions  px  and  M1?  for  each  test.  A  schematic  of  the  test  specimen  and  the  58  instrumented  locations  is  shown  in 
Fig.  3.  For  the  purposes  of  this  study,  the  analysis  is  limited  to  the  points  along  the  centerline  parallel  to  the  flow.  An 
investigation  by  Ostoich  et  al.4  discovered  that  the  recorded  data  at  points  1  and  38  may  have  been  affected  by  an 
uncharacterized  gap  between  the  dome  and  plate,  thus  only  the  middle  1 1  data  points  along  the  centerline  (points  2  - 
39)  are  considered. 
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Figure  3.  Spherical  dome  geometry  and  instrumented  locations  for  HTT  measurements5 


Table  1.  Experimental  conditions  from  Glass  and  Hunt  for  three  tests  at  different  dome  heights5 


Test 

Pi 

Mx 

WDl. 

w°)a 

Run  30 

654.9 

6.60 

0.028 

0.033 

Run  31 

648.0 

6.60 

0.013 

0.013 

Run  32 

645.9 

6.60 

0.006 

0.010 

The  flat  plate  and  spherical  dome  measurements  provide  substantial  validation  data  for  error  quantification  and 
confidence  assessments.  Furthermore,  since  the  domes  and  plate  are  thick  and  assumed  to  be  rigid,  potential 
coupling  between  structural  deformation  and  aerodynamic  heating  can  be  neglected.  Thus,  the  aerothermoelastic 
model  in  Fig.  2  simplifies  to  the  aerothermal  model  shown  in  Fig.  4. 


Figure  4.  a)  Aerodynamic  pressure  and  heating  models,  b)  Propagation  of  random  variables 


Figure  4a  shows  the  interaction  of  the  aerodynamic  pressure  and  heating  models  ,  and  Fig.  4b  shows  a  schematic 
of  uncertainty  propagation  for  the  primary  random  variables.  The  subscripts  of  the  flow  parameters  follow  the  same 
numbering  convention  as  defined  in  Fig.  1  for  the  representative  hypersonic  vehicle  forebody.  Location  ‘4’  refers  to 
the  location(s)  of  interest  on  the  flat  plate  (fp )  or  spherical  dome  ( sd ).  The  oblique  shock  relations  (Eqs.  (l)-(4))  give 
the  properties  of  the  inviscid  flow  (p3,  r3,  and  M3)  parallel  to  the  inclined  surface  as  a  function  of  the  freestream 
conditions  (p\,Tu  and  Mi),  shock  wave  angle  /?,  and  surface  inclination  angle  6 . 22 
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=  1  +  — (M,2  sin2  (jS)  - 1) 

Pi  r+ iv  y  ’  ’ 


(i) 


A_  (r  +  l)^i2sin2(y?) 

Pi  (r-l)M2sin2(^)  +  2 


TS  PjPl 

T1  A  /a 


M32  sin2  (yff-0) 


M12sin2(yg)  +  ^r 
(AK  sin2  (/?)-! 


(3) 

(4) 


Equation  (1)  is  used  to  calculate  the  aerodynamic  pressure  prediction  on  the  flat  plate,  pf  -  p3 .  For  a  deformed 
surface,  or  a  spherical  dome,  another  model  must  be  used  to  calculate  the  aerodynamic  pressure,  such  as  piston 
theory.3  A  3rd-order  expansion  of  piston  theory  (Eq.  (5))  is  used  in  this  study,  due  to  the  combined  presence  of 
hypersonic  flow  and  moderate  protrusion  of  the  spherical  domes  into  the  flow. 


p1 


3  M, 


r  1  dw  dw ^ 
v  U3  dt  dx  J 


+ 


y  +  \ 


r  1  dw 
v  U3  dt  dx  J 


+ 


y  +  1 
~Y1 


M 


1  dw 

v  U3  dt  dx  J 


(5) 


After  computing  the  inviscid  flow  over  the  flat  plate  or  spherical  dome,  the  heat  flux  can  be  calculated  using 
several  different  approaches.  In  a  higher-fidelity  model,  computational  fluid  dynamics  could  be  employed  to  predict 
both  the  aerodynamic  pressure  and  heating,  as  done  by  Ostoich  et  al.4  In  the  present  study,  a  more  computationally 
expedient  method  is  needed.  Eckert’s  reference  temperature  method  is  selected  because  it  provides  a  rapid 
approximation  for  the  boundary  layer  flow,  while  incorporating  local  inviscid  flow  properties  (location  ‘4’)  to 
capture  the  first  order  effect  of  panel  deformation  (spherical  dome  effect  in  this  case).  Using  flow  properties 
evaluated  at  Eckert’s  reference  temperature  (Eq.  (6))  the  aerodynamic  heat  flux  is  computed  using  Eq.  (7).3,23 


Te  +  0.5  (7;  -  Te )  +  0.22  (Taw  -  Te ) 

(6) 

Q4=St’pUecp(Taw-Tw ) 

(7) 

Where,  St  is  the  reference  Stanton  number,  p  is  the  reference  density,  Ue  is  the  inviscid  flow  velocity,  cp  is  the 

reference  specific  heat,  Taw  and  Tw  are  the  adiabatic  wall  and  actual  wall  temperatures,  respectively,  and  Te  is  the 
boundary  layer  edge  temperature.  Note  in  Eqs.  (6)  and  (7)  all  flow  properties  and  the  wall  temperature  are  evaluated 
at  the  point  of  interest  on  the  flat  plate  or  spherical  dome  (location  ‘4’). 

The  next  section  further  discusses  the  uncertain  inputs  in  the  aerothermal  model,  as  well  as  Bayesian  model 
parameter  calibration,  error  quantification,  and  validation  for  pressure  and  heat  flux  predictions. 

III.  Investigation  of  Model  Error  for  Aerothermal  Experiments 

This  section  analyzes  the  prediction  error  for  the  Glass  and  Hunt  experiments5  using  the  assumptions  and  results 
from  Culler  et  al.3  The  use  of  the  Glass  and  Hunt  data  in  this  paper  closely  follows  the  aerothermal  model 
verification  study  performed  in  Ref.  [3].  This  work  reevaluates  some  of  the  assumptions  and  errors  that  were 
observed  in  the  previous  study. 

First,  a  description  of  the  uncertain  input  parameters  in  the  experiments  and  aerodynamic  pressure  and  heating 
calculations  is  provided  with  sensitivity  analysis.  Next,  two  sets  of  experimental  data  are  used  to  calibrate  uncertain 
model  inputs  and  errors.  The  calibrated  inputs  are  then  used  to  update  nominal  predictions  for  the  spherical  dome 
experiments.  Then,  a  third  data  set  is  used  for  validation  with  Bayesian  hypothesis  testing-based  confidence.  Finally, 
a  model  selection  study  is  performed  using  the  confidence  metric  for  different  forms  of  piston  theory. 
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A.  Model  Input  Uncertainty  and  Sensitivity  Analysis 

Consider  the  flat  plate  specimen,  where  oblique  shock  relations  are  used  for  aerodynamic  pressure  pfp  and 
Eckert’s  reference  temperature  method  for  aerodynamic  heating  Qfp  .  Note  that  for  the  flat  plate,  we  are  interested  in 

the  value  at  the  center  of  the  plate,  which  corresponds  to  location  ‘4’  in  Fig.  1.  The  flat  plate  experiments  consisted 
of  three  tests  (Runs  30,  31,  and  32),  which  all  correspond  to  the  same  nominal  inputs  and  turbulent  boundary-layer 
with  a  sharp  leading  edge  panel  holder.  For  these  tests,  the  freestream  pressure  pu  and  Mach  number  Mu  were  given 
as  shown  in  Table  1.  In  addition,  the  output  aerodynamic  pressure  and  heat  flux  were  measured  at  the  center  of  the 
flat  plate.  However,  three  critical  pieces  of  information  were  not  available  in  the  Glass  and  Hunt5  report:  the 
freestream  temperature  Tx,  wall  temperature  Tw4 ,  and  equivalence  ratio  Req.  Therefore,  realistic  values  had  to  be 
estimated  from  other  reports  of  similar  testing.24  The  mean  freestream  and  wall  temperatures  are  assumed  to  be 
220K  and  300K,  respectively.  The  equivalence  ratio  is  also  uncertain,  but  for  the  current  investigation  a  constant 
value  of  Req  =  0.9  is  assumed. 

To  get  a  better  understanding  of  the  uncertainty  in  the  outputs  and  their  sensitivity  to  the  inputs,  statistical 
distributions  were  assumed  for  the  inputs.  Since  px  and  Mx  were  measured,  1%  coefficient  of  variation  (CV)  is  used 
for  measurement  variability.  However,  10%  CV  is  used  for  Tx  and  Tw4  since  they  were  not  reported  and  had  to  be 
assumed.  Normal  distributions  are  used  for  all  four  random  inputs  and  their  distribution  parameters  are  shown  in 
Table  2. 


T able  2.  Uncertainty  for  inputs  to  aerodynamic  pressure  and  heat  flux  calculations 


Measured  Inputs 

Mean 

Standard 

Deviation 

Coefficient  of 
Variation 

Px  (Pa) 

652.5 

6.525 

i% 

Mi 

6.6 

0.066 

i% 

Uncertain  Inputs 

Mean 

Standard 

Deviation 

Coefficient  of 
Variation 

T\  (K) 

220 

22.0 

10% 

TW4  (K) 

300 

30.0 

10% 

Local  and  global  sensitivity  analyses  are  performed  to  investigate  the  sensitivity  of  aerodynamic  pressure  and 
heating  to  the  input  variables.  For  defining  the  sensitivity  measures,  let  Y  -  f  (Xl,X2,...,Xn ) ,  where Xt  is  the 


measured  or  uncertain  inputs  and  Y  is  the  resulting  random  output.  The  local  sensitivity  is  calculated  as  the 
difference  of  the  total  variance  var(f),  to  the  variance  when  each  of  the  corresponding  random  variables  is  evaluated 
at  their  mean  with  the  other  inputs  remaining  random  (Eq.  (8)).21  The  greater  the  value  of  Act2  ,  the  greater  the 
importance  ofXt  on  Y.  Note  that  Al*  refers  to  being  calculated  over  all  random  variables  X,  except  A7;.  The  global 
sensitivity  is  expressed  as  main  effect  sensitivity  index  St  and  total  effect  sensitivity  index  STi  shown  in  Eqs.  (9)  and 
(10),  respectively.21  The  St  of  a  variable  is  another  measure  of  the  sensitivity  of  Xt  on  Y  and  STi  provides  information 
about  the  interaction  of  Xt  with  other  variables.  The  sensitivities  for  the  initial  random  inputs  in  Table  2  are  shown  in 
Table  3. 


Ao-2  =- 


var(r)-vaiy,;(r|X  =x,) 
var(T) 

varXi[£X:(F|X,.)] 


s.  = 


ST  = 


var(T) 

^,[vav(Fi  *-<)] 


var 


(r) 


(8) 

(9) 


(10) 
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Table  3.  Local  and  global  sensitivity  for  aerodynamic  pressure  and  heat  flux  for  the  flat  plate  with  initial 
_ uncertainty _ 


Input  Variable 

Kf 

sf 

sf 

Kf 

sf 

sf 

P\  (Pa) 

0.686 

0.684 

0.680 

0.020 

0.016 

0.016 

Mx 

0.333 

0.319 

0.319 

0.122 

0.174 

0.175 

T\  (K) 

0.0008 

0.0002 

0.0002 

0.451 

0.464 

0.465 

TW4  (K) 

- 

- 

- 

0.340 

0.344 

0.344 

As  expected,  the  temperatures  play  a  small  role  in  the  pf  calculation;  Tw4  does  not  appear  in  oblique  shock 
relations  and  Tx  is  only  used  with  Req  to  determine  the  methane-air  properties.  However,  Tx  and  Tw4  are  dominant  in 
the  heat  flux  calculation  with  0.451  and  0.340,  respectively.  Furthermore,  since  the  sum  of  the  main  effect  indices  St 
is  close  to  1,  individual  values  of  the  main  effect  indices  St  and  the  total  effect  indices  STi  are  so  similar,  it  is 
indicated  that  there  is  not  a  strong  interaction  among  variables.  Table  4  shows  the  forward  uncertainty  propagation 
of  the  normal  random  variables  from  Table  2  to  p  f  and  Q{p  . 


Table  4.  Uncertainty  propagation  using  initial  uncertainty  to  pressure  and  heat  flux  for  the  flat  plate 


Output 

Mean 

Coefficient  of 
Variation 

pf  (Pa) 

1385.4 

1.21% 

Qf  (  W/cm2 ) 

5.211 

11.53% 

Observe  that  the  10%  uncertainty  in  the  temperatures  (Tx  and  Tw4)  play  a  larger  role  in  the  Qfp  calculation, 

therefore  it  has  a  larger  CV  at  11.53%.  Since  these  experimental  values  are  unknown  and  the  distributions  are 
assumed,  it  is  beneficial  to  calibrate  these  uncertain  model  inputs.  The  next  section  uses  Bayesian  updating  to 
calibrate  the  Tj  and  Tw4  distributions  and  quantify  the  model  errors  using  a  Bayes  network  with  the  Glass  and  Hunt5 
data. 

B.  Bayesian  Model  Parameter  Calibration 

There  is  significant  epistemic  uncertainty  in  the  true  values  of  Tx  and  Tw4 ,  therefore  Bayesian  model  parameter 
calibration  can  assist  in  better  approximating  these  values  based  on  observations.  Furthermore,  the  errors  in 
aerodynamic  pressure  and  heat  flux  for  the  flat  plate  and  spherical  dome  predictions  can  also  be  calibrated.  First,  as 
a  brief  introduction  to  Bayesian  concepts,  let  (p  be  the  uncertain  model  parameters  or  errors  in  a  model  x(^)  with 

some  prior  information  on  the  parameters’  uncertainty  as  a  basis  for  a  statistical  distribution  n{qi)  .  Then  using 

some  observed  data  y,  the  distribution  of  the  unknown  parameters  is  updated  using  Bayes  theorem,  as  shown  in  Eq. 

(II).18 


|Pr|y|x(^)  n((p)d(p 


(11) 


Thus,  this  Bayesian  updating  reduces  the  uncertainty  in  the  parameters  (p ,  given  observations  y.  In  this  case,  the 

uncertain  parameters  are  (p  =  Tw4 ,  efA ,  ,  es^ ,  es£  ]  ;  where,  efA ,  ,  es^ ,  and  es£  are  the  errors  in  the 

aerodynamic  pressure  and  heat  flux  predictions  for  the  flat  plate  and  spherical  dome  specimens.  The  model  error  is 
defined  as  the  difference  between  the  model  prediction  and  the  true  value,  as  shown  in  Eqs.  (12)  and  (13)  for  the  flat 
plate  and  spherical  dome  models,  respectively.  For  this  study,  a  systematic  error  in  the  predictions  across  the  dome 
is  assumed  for  convenience,  as  seen  in  Eq.  (13). 
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(12) 


pI=pL+<  Qt=Qt+< 

(pt\  =(psL\+<  (Qt)=(Qt\+<  <13> 

Building  upon  the  relationship  of  the  inputs  and  predictions  from  Fig.  4b,  a  Bayes  network  of  the  measured 
inputs  (pi  and  Mx),  uncertain  inputs  (7\  and  Tw4),  model  predictions  (p4  and  Q4),  and  model  errors  (e  and  eQ^ )  is 

constructed.  Figure  5  depicts  the  Bayes  network  for  the  aerodynamic  pressure  and  heat  flux  predictions  for  the  flat 
plate  and  spherical  dome  geometries  and  the  interconnections  between  inputs,  errors,  and  data. 


The  gray  nodes  in  Fig.5  are  the  uncertain  inputs  and  errors  that  are  being  calibrated  with  the  Glass  and  Hunt5  data. 
The  dashed-box  around  the  network  represents  the  randomness  in  the  measured  inputs px  and  Mx.  Bayes  theorem  in 
Eq.  (11)  is  rewritten  for  the  case  corresponding  to  the  aerodynamic  pressure  and  heat  flux  predictions  for  the  Glass 
and  Hunt5  experiments  in  Eq.  (14). 


n[(p\yPi,yQ^y^yu) 


Pr 

[yPt  ’  >  yPt  >  +4  (<p)  .  Qa  W .  Pi> Mi  

}”{<p) 

H 

yPt  >  y&  >  yPl  >  yUl  1  p*  M  >  G*  (<p)  >  &>■ K  ] 

n[(p)d(p 

(14) 


In  Eq.  (14),  uncertain  inputs  and  errors  are  cp  =  ,  TwA ,  ,  es^ ,  J  ,  data  is  available  for  yf4  and  y^  from  the 

flat  plate  and  spherical  dome  measurements,  as  well  as  measured  input  data  y  and  yM  (Table  1).  Therefore,  all 

four  sources  of  data  are  incorporated  in  the  likelihood  function.  Now  that  q>  and  y  are  identified,  we  must  now 

define  the  prior  distributions  7i(yp)  .  Normal  distributions  are  used  for  uncertain  inputs  Tx  and  Tw4,  with  means  from 

Culler  et  al.3  and  10%  coefficient  of  variation,  as  summarized  in  Table  2.  Regarding  model  errors,  observations  from 
previous  reports  indicated  that  p4  and  Q4  predictions  are  expected  to  be  accurate  within  [-10%, +  10%]  and 

[-10%,  -30%] ,  respectively.24  The  error  bounds  for  Q4  are  associated  with  Eckert’s  reference  temperature  method, 

which  is  expected  to  consistently  over-predict  the  true  value  due  to  the  calorically  perfect  gas  assumption.  However, 
after  a  preliminary  comparison  of  predictions  to  data,  a  uniform  distribution  over  the  range  [-30%,  +30%]  of  the 
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prediction  was  determined  to  be  a  more  appropriate  prior  for  all  four  error  terms  in  this  study.  Therefore,  this  error 
model  assumes  uniform  distributions  based  on  the  experimental  means  for  the  prior  distribution  of  errors  n{(p) . 

Normal  distributions  are  used  for  the  likelihood  function  Pr[y  |x(^)]  ,  where  the  distribution  parameters  from 
Table  2  are  assumed  for  and  yM] | ,  whereas  5%  measurement  uncertainty  is  assumed  for  aerodynamic  pressure 
and  heat  flux  measurements  yf4  and  yQ^ . 

Bayesian  updating  according  to  Eq.  (14)  is  performed  using  all  of  the  observed  data  from  Glass  and  Hunt5, 
except  for  Run  30  for  the  spherical  dome.  Run  30  data  is  reserved  for  validation,  which  is  discussed  in  the  following 
section.  When  performing  the  Bayesian  updating,  the  freestream  pressure  pu  and  Mach  number  M1?  are  also  treated 
stochastically  due  to  the  measurement  uncertainty  presented  in  Table  2  with  1%  CV.  Equation  (14)  is  evaluated  at 
100  realizations  of  px  and  Mx  using  Latin  Hypercube  sampling.  For  each  of  those  samples,  a  Markov  Chain  Monte 
Carlo  (MCMC)  algorithm  called  slice  sampling  is  employed  using  104  samples  to  calculate  the  posterior 
distribution.  Figures  6  and  7  show  the  integrated  posterior  distributions  for  the  uncertain  inputs  and  errors 


Figure  6.  Prior  and  posterior  distributions  for  a)  freestream  temperature  Tx,  and  b)  wall  temperature  Tw4 
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Figure  7.  Prior  and  posterior  distributions  for  a)  error  in  flat  plate  p4,  b)  error  in  flat  plate  Q4 ,  c)  error  in 

spherical  dome  p4,  and  d)  error  in  spherical  dome  Q4 
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The  mean  and  standard  deviation  of  the  posterior  distributions  for  <p  =  ,  Tw4 ,  efpp ,  ,  es^ ,  es£  J  are  shown  in 

Table  5.  Comparing  the  initial  and  updated  distributions  of  Tx  and  Tw4 ,  it  is  seen  that  the  uncertainty  is  reduced, 
however  the  mean  value  did  not  shift.  This  is  primarily  a  result  of  the  errors  in  p4  and  Q4  predictions  being  more 
easily  scaled  as  defined  in  Eqs.  (12)  and  (13).  Thus,  calibrating  the  errors  did  result  in  a  shift  in  the  mean  values,  as 
seen  in  Fig.  7.  Also,  there  is  significant  uncertainty  reduction  in  the  errors  from  the  initial  ±30%  . 


Table  5.  Mean,  standard  deviation,  and  coefficient  of  variation  of  calibrated  model  inputs  and  errors 


Output 

Mean 

Standard 

Deviation 

Coefficient  of 
Variation 

T\  (K) 

220.67 

9.25 

4.19% 

TwA  (K) 

300.34 

28.24 

9.40% 

<  (Pa) 

-108.87  (-8.5%) 

40.36 

37.07% 

<  fc) 

-4707.0  (-10.0%) 

280.2 

5.95% 

<  (Pa) 

-75.84  (-5.0%) 

21.28 

28.06% 

<  (%.’) 

-4682.6  (-7.5%) 

247.8 

5.29% 

The  calibrated  distributions  for  Tx  and  Tw4  are  propagated  to  pfp  and  Q{p  in  Table  6.  The  uncertainty  in 
aerodynamic  pressure  is  unchanged  since  it  is  insensitive  to  freestream  temperature.  However,  the  uncertainty  in 
Q{p  is  reduced  from  1 1.53%  to  6.46%. 


Table  6.  Uncertainty  propagation  using  updated  uncertainty  to  pressure  and  heat  flux  for  the  flat  plate 


Output 

Mean 

Coefficient  of 
Variation 

P?  (Pa) 

1385.4 

1.23% 

Q{p  (w/ cm2 ) 

5.229 

6.46% 

The  next  section  investigates  the  effect  of  quantifying  the  model  errors  in  the  predictions  for  the  spherical  dome 
and  uses  the  remaining  set  of  data  (Run  30)  for  assessing  the  confidence  in  pf  and  Qf  predictions. 

C.  Assessing  Prediction  Confidence  for  Model  Validation 

The  aerodynamic  pressure  and  heat  flux  along  the  spherical  dome  is  evaluated  at  the  initial  and  updated  values  of 
<p  =  \TxJwA,esd^eSQA  ]  from  Table  5.  Figures  8  and  9  show  the  experimental  data  from  Glass  and  Hunt5  (Runs  30,  31, 

and  32),  along  with  the  initial  and  updated  model  predictions  pf  and  Qf  evaluated  at  the  mean  along  the 

streamwise  centerline  of  the  spherical  domes  (see  Fig.  3).  Tables  7  and  8  summarize  the  deterministic  errors  in  the 
predictions.  As  illustrated  in  Figs.  8  and  9,  aerodynamic  pressure  and  heating  are  greatest  near  the  leading  edge  of 
the  dome  and  lowest  near  the  trailing  edge.  This  is  a  result  of  the  slope  of  the  dome  in  the  flow  direction,  where 
positive  slope  results  in  elevated  values  and  negative  slope  produces  lower  values  relative  to  the  flat  plate.  Thus,  the 
largest  dome  (Run  30)  produces  the  greatest  spatial  variations  in  pressure  and  heating.  Note  that  the  slope  of  each 
dome  is  zero  at  x/D= 0.5.  At  this  location,  pressure  and  heating  values  are  nearly  identical  for  each  dome  and  for  the 
flat  plate,  which  indicates  that  local  surface  inclination  has  a  strong  impact  on  local  pressure  and  heating  values. 
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Figure  8.  Aerodynamic  pressure  along  centerline  of  spherical  dome  for  Runs  30,  31,  and  32  from 
experimental  data,  initial  mean  input  values,  and  Bayesian  updated  mean  input  values 


Table  7.  Error  summary  for  nominal  pressure  predictions  along  centerline  of  spherical  dome 


Initial  Errors  in  pf 

Updated  Errors  in  pf 

Run  30 

Run  31 

Run  32 

Run  30 

Run  31 

Run  32 

Average 

13.6% 

8.2% 

6.7% 

16.0% 

8.3% 

2.7% 

Maximum 

39.3% 

23.5% 

14.2% 

35.4% 

18.5% 

8.9% 

From  Fig.  8  and  Table  7  it  is  evident  that  3rd-order  piston  theory  predictions  of  pf  become  less  accurate  with 
increasing  dome  surface  inclination.  Accordingly,  the  largest  error  in  pf  occurs  at  the  forward-most  location  in 

Run  30.  Recall  that  Run  30  was  saved  for  validation  and  only  Runs  31  and  32  were  included  in  calibration.  This 
generally  resulted  in  smaller  errors  for  Runs  31  and  32,  but  errors  for  Run  30  increased,  as  summarized  in  Table  7.  It 
is  expected  that  if  data  from  Run  30  had  been  included  in  calibration,  then  the  corresponding  errors  would  have  also 
been  reduced.  Furthermore,  since  the  errors  in  pf  along  the  dome  vary  in  magnitude  spatially,  it  would  be 
beneficial  to  use  a  more  flexible  error  model,  such  as  a  Gaussian  process  model,  in  more  practical  applications.26 

Figure  9  and  Table  8  show  that  Eckert’s  reference  temperature  method  predicted  Qf  more  accurately  than  3rd- 
order  piston  theory  predicted  pf  .  Again,  the  trend  is  observed  that  errors  are  reduced  using  the  updated  q>  values 
for  Runs  31  and  32,  but  not  Run  30.  This  may  be  expected  since  values  of  yf  from  Runs  31  and  32  are  included  in 
the  Bayesian  calibration,  whereas  values  from  Run  30  were  not  used. 
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Figure  9.  Aerodynamic  heat  flux  along  centerline  of  spherical  dome  for  Runs  30,  31,  and  32  from 
experimental  data,  initial  mean  input  values,  and  Bayesian  updated  mean  input  values 


Table  8.  Error  summary  for  nominal  heat  flux  predictions  along  centerline  of  spherical  dome 


Initial  Errors  -  Qf 

Updated  Errors  -  Qf 

Run  30 

Run  31 

Run  32 

Run  30 

Run  31 

Run  32 

Average 

3.9% 

6.2% 

12.8% 

11.7% 

4.1% 

3.6% 

Maximum 

12.9% 

10.4% 

21.2% 

24.7% 

8.7% 

12.4% 

The  deterministic  errors  are  useful  for  assessing  the  accuracy  of  the  nominal  model  predictions,  however  this  is  a 
stochastic  problem  and  error  alone  does  not  provide  a  statistical  assessment  of  the  confidence  in  the  model 
prediction.  Therefore,  the  most  important  step  in  this  model  uncertainty  framework  is  to  validate  the  models  by 
assessing  the  confidence.  This  enables  decision-making  in  regard  to  model  development  and  fidelity  selection.  For 
the  Aircraft  Digital  Twin,  it  is  important  to  have  this  confidence  metric  to  make  autonomous  decision  making 
possible  for  efficient  simulations  and  risk  mitigation. 

Several  validation  metrics  exist  with  advantages  and  disadvantages,  such  as  classical  hypothesis  testing,  and 
difference  and  area  metrics;  however  Bayesian  hypothesis  testing  is  selected  for  this  study.18,27,28  The  Bayes  factor 
approach  fits  appropriately  with  the  Bayes  network  integration  framework,  but  its  main  advantages  are  that  it  takes 
into  account  the  entire  probability  distribution  of  the  model  output  and  its  relation  to  a  confidence  metric  is 
straightforward.  For  Bayesian  hypothesis  testing,  we  want  to  determine  the  probability  of  our  model  being  correct, 
given  some  observed  data.  Consider  a  hypothesis  test  to  determine  the  probability  that  a  model  prediction  x  is  equal 
to  its  true  value  x0.  Equation  (15)  calculates  the  Bayes  factor  B ,  as  the  ratio  of  likelihoods  corresponding  to  the  null 
hypothesis  (model  prediction  is  equal  to  the  true  value)  and  the  alternate  hypothesis  (model  prediction  is  not  equal  to 
the  true  value).  Therefore,  when  B  >  1,  the  data  supports  the  null  hypothesis  better  than  the  alternative  hypothesis. 
The  integral  form  of  the  Bayes  factor  in  Eq.  (15)  includes  the  likelihood  function  of  the  data  supporting  the 
prediction  Pr(y]x),  the  probability  density  function  (PDF)  of  the  model  prediction  7r0  (x)  ,  and  the  PDF  for  the 

alternative  hypothesis  nx  (x) . 


Pr(^  \H0\x  =  x0)  JPr(y|xK(x)<ifr 
°  Pr(j;  \H\ :  x  *  x0)  jPr(_y|x)^1(x)(fe 


(15) 
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Equation  (15)  is  rewritten  in  Eqs.  (16)  and  (17)  for  the  cases  for  aerodynamic  pressure  and  heat  flux,  where  i=  1  to 
1 1  for  the  x-location  across  the  dome. 


|Pr| 

[y  eg  I  , 

1*0  (Qt)dQt 

|Pr| 

k  iar! 

\^(Qt)dQt 

(16) 


(17) 


The  Bayes  factor  is  calculated  at  each  of  the  11  points  along  the  spherical  dome  used  in  Run  30  of  the  Glass  and 
Hunt5  study.  The  likelihood  function  Pr(y|x)  is  based  on  the  assumption  of  a  normal  distribution  for  measurement 
error,  with  a  standard  deviation  based  on  5%  coefficient  of  variation  on  the  mean  of  the  pressure  and  heat  flux  data 
for  Run  30  ( 7V(y?.,0.05y) ).  The  probability  density  function  for  the  null  hypothesis  ;r0(x),  is  determined  by 

propagating  the  uncertainty  in p\,  Mh  Tu  and  Tw4,  as  well  as  the  quantified  errors  for  the  spherical  dome  ( es^  and 

eSg4 ).  The  PDF  for  the  alternative  hypothesis  nx  (x) ,  is  modeled  as  a  uniform  distribution  extending  beyond  the 

maximum  and  minimum  values  of  pf  and  Qf  predictions. 

The  Bayes  factors  computed  in  Eqs.  (16)  and  (17)  can  be  used  to  the  confidence  C,  in  the  prediction,  as  shown  in 
Eq.  (18).25 


c=^i=p(H°\y)  (18> 

As  indicated  in  Eq.  (18),  C  is  simply  the  posterior  probability  of  the  null  hypothesis  being  true,  given  the 
observation  data  (under  the  assumption  that  prior  probabilities  of  the  null  and  alternative  hypotheses  are  both  0.5). 
For  a  Bayes  factor  of  1.0,  the  confidence  C,  is  equal  to  50%.  This  implies  that  we  do  not  have  enough  evidence  to 
reject  or  accept  the  null  hypothesis.  However,  for  Bayes  factors  greater  than  1.0  (as  explained  for  Eq.  (15)),  we 
would  have  increasing  confidence  that  the  prediction  is  equal  to  the  true  value.  The  confidence  metric  can  be  used  as 
a  resource  allocation  measure  for  determining  when  it  is  beneficial  to  perform  tests,  where  higher  fidelity  models  are 
required,  and  which  disciplines  need  a  more  strongly  (or  less  strongly)  coupled  solution  procedure.  In  addition,  the 
Bayes  factor-based  confidence  can  be  used  to  assess  the  limits  of  the  model’s  predictions. 

Table  9  summarizes  the  confidence  (Eqs.  (16)-(18))  and  deterministic  error  in  pf  and  Qf  predictions  for  the 
1 1  observations  from  Run  30  (y  sd  and  y„d ). 

Pm  Qm 
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Table  9.  Error  and  confidence  in  aerodynamic  pressure  and  heat  flux  predictions  along  centerline  of 


Location 

x/D 

%error 

pf 

Cpi 

% error 

Qf 

c  sd 

^4  i 

1 

0.11 

-35.4% 

0.00% 

2.5% 

86.7% 

2 

0.19 

-21.0% 

0.02% 

2.8% 

87.5% 

3 

0.26 

-9.3% 

71.1% 

3.9% 

87.7% 

4 

0.34 

-3.9% 

94.0% 

4.4% 

88.6% 

5 

0.42 

2.4% 

94.6% 

8.4% 

85.1% 

6 

0.50 

7.0% 

88.7% 

9.7% 

85.0% 

7 

0.58 

11.3% 

74.8% 

14.5% 

74.1% 

8 

0.66 

15.3% 

54.9% 

21.3% 

36.1% 

9 

0.74 

21.4% 

16.4% 

24.7% 

25.0% 

10 

0.81 

23.7% 

21.7% 

17.1% 

81.8% 

11 

0.89 

25.9% 

30.2% 

19.7% 

81.5% 

The  majority  of  the  predictions  have  greater  than  50%  confidence,  which  means  the  data  supports  the  prediction. 
Pressure  predictions  at  locations  4-6  have  the  highest  confidence.  The  confidence  in  Qf  predictions  using  Eckert’s 
reference  temperature  method  are  all  above  80%,  with  the  exception  of  locations  7-9.  As  mentioned  for  Fig.  8,  the 
largest  error  in  the  pf  occurs  at  the  front  of  the  dome,  which  corresponds  to  0%  confidence.  The  deterministic 
errors  give  an  indication  of  the  quality  of  the  nominal  predictions,  however  note  that  it  is  not  always  indicative  of  the 
statistical  confidence  in  the  predicted  values.  For  example,  2.4%  error  at  location  5  for  pf  corresponds  to  94.6% 
confidence,  but  2.5%  error  at  location  1  for  Qf  is  lower  at  86.7%  confidence.  This  is  the  result  of  differences  in  the 
shape  of  the  model  error  distributions  for  pf  and  Qf  . 

The  scope  of  this  work  is  to  initiate  a  framework  for  assessing  the  confidence  in  coupled  aerothermoelastic 
model  predictions,  not  to  necessarily  draw  definitive  conclusions  for  these  particular  aerodynamic  models.  However, 
given  the  confidence  associated  with  the  pf  predictions  for  Run  30  {HID  =  0.028),  one  would  likely  conclude  that 
3rd-order  piston  theory  is  inadequate  for  predicting  the  aerodynamic  pressure  on  a  spherical  dome  protuberance  of 
this  size.  Although,  to  truly  reach  that  conclusion,  more  thorough  uncertainty  quantification  is  required  to  better 
capture  the  model  error. 

The  final  section  builds  upon  the  conclusions  using  the  Bayes  factor-based  confidence  and  uses  that  metric  to 
compare  predictions  using  different  forms  of  piston  theory. 

D.  Model  Selection  from  Prediction  Confidence  Metric 

The  Bayes  factor-based  confidence  metric  is  also  useful  for  model  selection.  Up  to  this  point,  a  3rd-order 
expansion  of  piston  theory  from  Eq.  (5)  was  used  to  predict  pf  .  Naturally,  1st-  and  2nd-order  expansions  could  have 
been  used  instead.  Consider  1st-  and  2nd-order  piston  theories  for  a  model  selection  study,  as  shown  in  Eqs.  (19)  and 
(20),  respectively. 


f  1  dw  dw 
yU3  dt  dx  j 


(19) 


pf=p3+  2-J- 
M, 


^  1  dw  dw ^ 
KU3  dt  dx  j 


r+i , . 
+  - M, 


^  1  dw  dw ^ 
KU3  dt  dx  j 


(20) 
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Figures  10  and  11  show  the  pf  and  Qf  predictions  for  Run  30  using  1st-,  2nd-,  and  3rd-order  piston  theories. 
Recall  that  Eckert’s  reference  temperature  method  uses  the  pf  predictions  from  piston  theory,  so  Qf  is  also 
affected. 
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Figure  10.  Aerodynamic  pressure  predictions  for  Run  30  using  1st-, 2nd-,  and  3rd-order  piston  theory 


Figure  11.  Aerodynamic  heat  flux  predictions  for  Run  30  using  1st-, 2nd-,  and  3rd-order  piston  theory 


Comparing  the  different  piston  theories,  both  2nd-  and  3rd-order  capture  some  of  the  nonlinearity  in  pf  and 
Qf  and  give  relatively  similar  predictions.  First-order  piston  theory  appears  to  give  a  more  accurate  prediction  at  the 
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front  of  the  dome  for  pf  ,  however  the  aft  portion  has  larger  errors  than  2nd-  and  3rd-order.  Furthermore,  the  1st- 
order  expansion  decreases  the  overall  accuracy  over  the  entire  dome  for  heat  flux  predictions. 

Table  10  shows  the  Bayesian  hypothesis  testing-based  confidence  metric  for  pf  and  Qf  using  1st-,  2nd-,  and 
3rd-order  piston  theory.  As  expected  from  Fig.  10,  lst-order  piston  theory  has  higher  confidence  at  the  front  of  the 
pressure  dome,  but  is  significantly  lower  when  compared  to  2nd-  and  3rd-order  predictions  along  the  centerline  of  the 
dome.  When  comparing  the  confidence  in  2nd-  and  3rd-order  pf  and  Qf ,  not  only  are  the  two  predictions  very 
similar,  but  the  confidence  in  the  2nd-order  model  is  actually  higher  at  locations  8-11.  Therefore,  not  only  would  a 
lower-order  model  theoretically  require  lower  computational  costs,  but  it  is  statistically  more  representative  of 
observations  along  the  dome  for  Run  30. 


Table  10.  Confidence  in  aerodynamic  pressure  and  heat  flux  predictions  using  1st-,  2nd-,  and  3rd-order  piston 

theory  along  centerline  of  spherical  dome  for  Run  30 


Location 

x/D 

C- 

P4i 

r 

lst-order 

2nd-order 

3rd-order 

lst-order 

2nd-order 

3rd-order 

1 

0.11 

0.36% 

0.00% 

0.00% 

31.5% 

84.8% 

86.7% 

2 

0.19 

63.1% 

0.09% 

0.02% 

62.2% 

86.7% 

87.5% 

3 

0.26 

94.4% 

75.6% 

71.1% 

77.8% 

87.5% 

87.7% 

4 

0.34 

95.4% 

94.0% 

94.0% 

85.7% 

88.5% 

88.6% 

5 

0.42 

94.2% 

94.7% 

94.6% 

83.8% 

85.2% 

85.1% 

6 

0.50 

89.0% 

89.1% 

88.7% 

84.7% 

85.0% 

85.0% 

7 

0.58 

68.6% 

74.9% 

74.8% 

69.3% 

73.8% 

74.1% 

8 

0.66 

20.3% 

57.1% 

54.9% 

10.6% 

38.3% 

36.1% 

9 

0.74 

0.16% 

23.3% 

16.4% 

0.25% 

35.0% 

25.0% 

10 

0.81 

0.00% 

42.7% 

21.7% 

0.23% 

88.8% 

81.8% 

11 

0.89 

0.00% 

69.0% 

30.2% 

0.00% 

92.0% 

81.5% 

IV.  Summary 

A  framework  to  quantify  the  model  error  and  assess  the  confidence  in  model  predictions  for  a  coupled 
aerothermoelastic  panel  is  outlined.  Bayesian  model  calibration,  error  quantification,  and  prediction  confidence 
assessment  procedures  are  described  for  aerothermal  models  with  data  available  from  tests  performed  in  a  High- 
Temperature  Tunnel  on  spherical  dome  protuberances  subjected  to  hypersonic  flow.  The  models  include  3rd-order 
expansion  of  piston  theory  and  Eckert’s  reference  temperature  method  to  predict  aerodynamic  pressure  and  heat 
flux,  respectively.  This  research  aims  to  logically  and  optimally  use  the  limited  data  available  for  model  validation 
and  decision-making.  The  freestream  and  wall  temperatures  are  assumed  since  their  values  were  not  reported  in  the 
experiments.  Bayesian  calibration  is  employed  to  update  the  uncertain  inputs  and  quantify  errors  associated  with 
aerodynamic  pressure  and  heat  flux  predictions.  The  calibrated  input  distributions  and  quantified  model  errors  are 
used  to  update  the  model  predictions  along  the  centerline  of  a  spherical  dome  specimen.  The  information  on  the 
model  error  is  used  to  calculate  the  Bayesian  hypothesis  testing-based  confidence  to  enable  model  validation  and 
model  selection  for  this  aerothermal  problem.  For  this  model  selection  study  among  piston  theories,  it  was  observed 
that  the  highest-order  model  (3rd-order)  did  not  result  in  the  highest  prediction  confidence  metric.  The  capability  to 
have  a  metric  for  autonomously  making  decisions  on  aerothermoelastic  model  fidelity  is  critical  for  the  USAF 
Digital  Twin  vision,  and  this  study  is  aimed  at  taking  steps  to  achieve  this  goal. 
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ADDENDUM:  Lab  Task  Summary 

Quantifying  Confidence  in  Model  Predictions  for 
Hypersonic  Aircraft  Structures 
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Lack  of  confidence  in  structural  response  and  life  predictions  of  a  vehicle  exposed  to 
combined  extreme  environments  has  consistently  prevented  the  USAF  from  fielding  affordable, 
reliable,  and  reusable  hypersonic  space  access  platforms.  Significant  strides  have  been  made  in 
modeling  complex  interactions  of  the  multi-physics,  fluid-thermal-structural  coupling  applicable 
to  hypersonic  flow  conditions.  However,  validation  of  these  models  remains  a  challenge  due  to 
limited  experimental  data  for  hypersonic  conditions.  This  research  addresses  fundamental  and 
critical  issues  in  quantifying  uncertainty  and  assessing  the  confidence  in  model  predictions  of 
hypersonic  structural  response  through  a  systematic  framework.  The  framework  will  provide  the 
capability  to  quantify  the  model  prediction  confidence,  enabling  its  use  as  a  decision-making 
metric  for  model  fidelity  selection  and  determining  the  necessary  level  of  coupling  in  the 
modeling  of  an  aerothermoelastic  system.  The  first  phase  of  this  research  effort  focused  on 
developing  a  model  uncertainty  framework  for  the  aerothermal  components  of  the  coupled 
system.  The  modeled  conditions  correspond  to  aerothermal  test  data  from  hypersonic  wind 
tunnel  experiments  of  a  spherical  dome  protuberance  on  a  flat  plate.  Local  and  global  sensitivity 
analyses  were  employed  to  investigate  the  sensitivity  of  model  predictions  to  the  input  variables. 
A  Bayes  network  was  constructed  for  aerodynamic  pressure  and  heating  to  integrate  model 
predictions,  aerothermal  data,  and  various  sources  of  uncertainty.  Bayesian  model  calibration 
was  performed  with  the  test  data  to  quantify  uncertain  model  parameters  and  errors.  Finally,  a 
Bayesian  hypothesis  testing-based  confidence  metric  was  used  to  compare  the  accuracy  of 
aerodynamic  pressure  predictions  for  different  forms  of  piston  theory. 
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